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Abstract 


The  objective  of  this  demonstration  was  to  determine  if  laser-positioned,  high-density  EM61  data 
acquired  in  a  moving  survey  mode  could  support  feature-based  discrimination  decisions  for  a 
canopied  site  in  Bridgeport  Connecticut.  Inversion  results  were  benchmarked  using  cued  data 
acquired  in  a  controlled,  gridded  approach  over  40  targets.  In  support  of  this  effort, 
electromagnetic  and  magnetic  data  were  acquired  using  man-portable  sensor  suites. 

The  geophysical  data  were  spatially  registered  using  a  robotic  total  station  because  canopy  at  the 
site  severely  restricts  the  use  of  differential  global  positioning  satellite  systems.  The  pitch,  roll 
and  three-axis  acceleration  measurements  of  the  sensors  were  measured  using  a  low-metal 
inertial  measurement  unit.  The  survey  site  was  seeded  with  inert  UXO,  ranging  in  size  from 
37mm  to  105mm  projectiles. 

The  feature -based  EMI  characterization  approach  demonstrated  here  inverts  measured  EM61 
field  data  assuming  a  point  dipole  source  and  then  segments  the  targets  by  feeding  the  inverted 
model  parameters  into  statistical  classifiers.  We  compared  parameter  estimates  derived  from 
both  data  sets  and  evaluated  overall  discrimination  performances. 

Results  indicate  that  the  laser-positioned  EM61  data  were  not  of  sufficient  quality  to  discriminate 
UXO  from  non-UXO.  Of  the  58  seeded  items  that  were  surveyed,  41  were  ranked  as  high 
confidence  ordnance,  10  were  ranked  as  high  confidence  clutter,  and  7  remained  unclassified. 
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Feature-based  Detection  and  Discrimination  at  DuPont’s 
Lake  Success  Business  Park,  Connecticut 

1.  Introduction 


1.1  Background 

With  current  technology  and  survey  practices,  unexploded  ordnance  (UXO)  site  characterization 
is  inefficient  and  incomplete.  Not  all  buried  UXO  are  routinely  detected,  and  UXO  cannot  be 
routinely  distinguished  from  other  items  in  the  ground  that  pose  no  risk.  Over  the  past  five  or  six 
years,  SERDP  and  ESTCP  have  invested  heavily  in  developing  survey  data  analysis  and 
processing  techniques  for  use  with  commercial  sensors  that  can  improve  UXO  detection  and 
discrimination  between  UXO  and  clutter.  These  techniques  include  characterization  procedures 
for  estimating  target  features  from  survey  data  (size,  shape,  depth  of  burial,  orientation,  etc.)  and 
feature-based  classification  procedures  to  aid  decision-making. 

Feature-based  characterization  and  classification  schemes  have  improved  discrimination 
performance  in  some  demonstrations.  These  algorithms,  however,  are  not  readily  available  to 
the  user  community  and  have  had  limited  exposure  to  data  acquired  under  'production-imposed' 
constraints. 

Our  technical  approach  promotes  the  selection  of  potential  UXO  targets  using  quantitative 
evaluation  criteria  and  transparent  decision-making  processes.  In  preparation  of  this 
demonstration,  we  developed  an  analysis  framework  within  Oasis  montaj™.  The  analysis 
algorithms  provide  quantitative  evaluation  criteria  (e.g.,  target  characterization  and 
classification).  Transparency  is  being  achieved  by  leveraging  the  professional,  flexible,  and 
visual  computing  environment  inherent  in  Oasis  montaj™.  Oasis  montaj™  is  a  geophysical  data 
processing  and  visualization  package  developed  and  marketed  by  Geosoft  Incorporated.  Oasis 
montaj™  has  a  large  capacity  database,  a  professional  graphic  interface,  and  an  established  client 
base. 


1.2  Objectives  of  the  Demonstration 

The  objective  of  this  demonstration  is  to  compare  discrimination  performance  using  laser- 
positioned,  high-density  EM61  data  acquired  in  a  moving  survey  mode  with  data  acquired  in  a 
stationary  grid  (cued)  approach  for  a  canopied  site,  DuPont's  Lake  Success  Business  Park 
(LSBP),  in  Bridgeport  Connecticut.  Sky  Research  acquired  electromagnetic  and  magnetic  data 
using  man-towed  systems,  one  of  which  leverages  work  associated  with  SERDP  program  UX- 
1310  entitled  ‘Sensor  Orientation  Effects  on  UXO  Geophysical  Target  Discrimination.’  AETC 
Incorporated  analyzed  the  data  using  UX-Analyze.  Results  from  this  Demonstration  are 
compared  and  contrasted  with  baseline  data  including  a  survey  conducted  by  a  commercial 
contractor  and  data  acquired  in  a  cued  fashion  (viz.,  stationary  sensor  over  fixed  location). 
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1.3  Regulatory  Drivers 


The  Senate  Report  (Report  106-50),  pages  291-293,  accompanying  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2000  (Public  Law  106-65),  included  a  provision  entitled 
“Research  and  development  to  support  unexploded  ordnance  clearance,  active  range  unexploded 
ordnance  clearance,  and  explosive  ordnance  disposal.”  This  provision  requires  the  Secretary  of 
Defense  to  submit  to  the  Congressional  defense  committees  a  report  that  gives  a  complete 
estimate  of  the  current  and  projected  costs,  to  include  funding  shortfalls,  for  UXO  response  at 
active  facilities,  installations  subject  to  base  realignment  and  closure  (BRAC),  and  formerly  used 
defense  sites  (FIJDS). 

The  following  statements  are  taken  verbatim  out  of  the  DoDs  2001  Report  to  Congress: 

“ Decades  of  military  training,  exercises,  and  testing  of  weapons  systems  has  required 
that  we  begin  to  focus  our  response  on  the  challenges  of  UXO.  Land  acreage 
potentially  containing  UXO  has  grown  to  include  active  military  sites  and  land 
transferring  or  transferred  for  private  use,  such  as  Base  Realignment  and  Closure 
(BRAC)  sites  and  Formerly  Used  Defense  Sites  (FUDS).  DoD  responsibilities  include 
protecting  personnel  and  the  public  from  explosive  safety  hazards;  UXO  site  cleanup 
project  management;  ensuring  compliance  with  federal,  state,  and  local  laws  and 
environmental  regulations;  assumption  of  liability;  and  appropriate  interactions  with 
the  public. 

...Through  limited  experience  gained  in  executing  these  activities,  it  has  become 
increasingly  clear  that  the  full  size  and  extent  of  the  impact  of  sites  containing  UXO 
is  yet  to  be  realized.  ...  DoD  has  completed  an  initial  baseline  estimate  for  UXO 
remediation  cost.  This  report  provides  a  UXO  response  estimate  in  a  range  between 
$106.9  billion  and  $391  billion  in  current  year  [2001]  dollars.  ...Technology 
discovery,  development,  and  commercialization  offers  some  hope  that  the  cost  range 
can  be  decreased.  ... 

...  Objective:  Develop  standards  and  protocols  for  navigation,  geo-location,  data 
acquisition  and  processing,  and  performance  of  UXO  technologies. 

•  Standard,  high  quality  archived  data  are  needed  for  optimal  data  processing  of 
geophysical  data,  re-acquisition  for  response  activities,  quality  assurance,  quality 
control,  and  review  by  all  stakeholders.  In  addition  standards  and  protocols  are 
required  for  evaluating  UXO  technology  performance  to  aid  in  selecting  the  most 
effective  technologies  for  individual  sites. 

•  Standard  software  and  visualization  tools  are  needed  to  provide  regulatory  and 
public  visibility  to  and  understanding  of  the  analysis  and  decision  process  made  in 
response  activities.  ” 
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1.3.1  Regulatory  Background  Specific  to  Lake  Success  Business  Park  (LSBP). 

LSBP  is  an  interim  status  Resource  Conservation  and  Recovery  Act  (RCRA)  facility.  On 
August  21,  1990,  Remington  Arms  Company,  Inc.  entered  into  an  Administrative  Consent  Order 
(ACO)  with  USEPA.  The  ACO  has  been  modified  twice.  The  first  modification  (1994) 
authorized  the  construction  of  a  Corrective  Action  Management  Unit  (CAMU),  which  is  a 
facility  designed  to  accumulate  and  treat  contaminated  soils.  The  second  modification  (1997) 
approved  the  implementation  or  Remedy  I  (excavation  and  treatment  of  upland  soils). 

A  Phase  I  RCRA  Facility  Investigation  (RFI)  was  performed  in  1989-1991  and  approved  by  EPA 
in  1992.  Fifty-one  Areas  of  Environmental  Concern  (AECs)  were  identified  and  investigated. 
The  primary  contaminant  of  concern  is  lead. 


1.4  Stakeholder/End-User  Issues 

The  stakeholders  and  end-users  of  this  target  characterization  and  classification  technology 
include  private  contractors  that  conduct  geophysical  investigations  in  support  of  UXO  clean  up 
programs  and  governmental  employees  that  provide  technical  oversight.  DuPont  is  a  key 
stakeholder.  The  demonstration  of  discrimination  capabilities  using  survey  data  will 
significantly  affect  future  clearance  activities  at  the  site. 
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2.  Technology  Description 


2.1  Technology  Developments  and  Application 

This  demonstration  includes  data  acquisition  and  analysis  components.  The  sensor  system  and 
platform  are  leveraged  from  SERDP  project  1310  entitled  ‘Sensor  orientation  effects  on  UXO 
geophysical  target  discrimination.’ 

2.1.1  UX- Analyze 

Primary  objectives  of  project  UX-200210  are  to  transfer  anomaly  characterization  and 
classification  algorithms  to  the  user  community  via  Oasis  montaj™  and  to  demonstrate  the 
efficacy  of  the  feature-based  approach  through  multiple  demonstrations  at  sites  with  a  variety  of 
near  surface  conditions  and  target  objects.  The  anomaly  characterization  algorithms  developed 
by  AETC  during  the  past  decade  assume  a  dipolar  source  and  derive  the  best  set  of  induced 
dipole  model  parameters  that  account  for  the  spatial  variation  of  the  signal  as  the  sensor  is  moved 
over  the  object.  Algorithms  for  magnetic  and  electromagnetic  induction  (EMI)  data  have  been 
developed  and  tested  during  previous  demonstrations.  The  model  parameters  are  target  X,Y 
location  and  depth,  three  dipole  response  coefficients  corresponding  to  the  principle  axes  of  the 
target  (EMI  only),  and  the  three  angles  that  describe  the  orientation  of  the  target.  The  size  of  the 
target  can  be  estimated  using  empirical  relationships  between  either  the  dipole  moment  for 
magnetic  data  or  the  sum  of  the  targets’  response  coefficients.  Cylindrical  objects  like  most 
UXO  have  one  large  coefficient  and  two  smaller,  equal  coefficients.  Plate-like  objects  nominally 
have  two  large  and  one  small  coefficient. 

UX-Analyze  was  developed  during  the  first  year  of  the  program  to  facilitate  efficient  UXO  data 
analysis  within  the  Oasis  montaj™  environment.  It  consists  of  multiple  databases,  custom 
graphical  interfaces,  and  data  visualizations.  UX-Analyze  provides  the  infrastructure  to 
systematically  identify  and  extract  anomalies  from  the  dataset,  call  the  characterization  routines, 
store  the  fitted  source  parameters  for  each  anomaly,  perform  target  classification,  and  document 
the  analysis.  Once  the  analysis  is  complete,  individual  images  for  each  anomaly  can  be 
automatically  produced  for  documentation  purposes. 

2.1.2  Hardware 

Sky  Research  deployed  a  G-858  magnetometer,  made  by  Geometries,  and  an  EM61  EMI  system, 
made  by  Geonics  Ltd.,  at  the  LSBP  site  (Figure  1).  Four  G-858  sensors  were  deployed  using  a 
rigid  fiberglass  push  cart  system.  The  EM61  was  deployed  using  the  standard  wheel/coil 
configuration  provided  by  Geonics.  The  magnetic  and  EM  data  were  position-stamped  with  data 
collected  from  the  Leica  robotic  total  station  (RTS).  The  orientation  of  the  sensors  during 
collection  was  measured  using  a  low-metal  inertial  measurement  unit  (IMU)  providing  pitch,  roll 
and  3-axis  acceleration  measurements. 
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Figure  1.  (Left)  The  standard  EM61  /  RTS  pushcart  configuration  was  modified  for  the  DuPont 
survey  by  adding  the  low-metal  IMU  and  lowering  the  prism  height.  (Right)  Photograph  of  the 
pushcart  magnetometer  /  RTS  system. 

The  following  component  technologies  were  used  as  part  of  the  LSBP  surveys. 

•  Gyro:  RIS  Corporation  fabricated  a  low-metal  IMU  sensor  for  measurement  of  pitch,  roll 
and  three-axis  acceleration.  The  unit  has  very  low  metal  components.  The  unit  has  a  built-in 
sensor  that  allows  the  measurement  to  be  interleaved  with  the  EM  firing  sequence.  The  gyro 
was  housed  in  a  fiberglass  box  and  has  plastic  connectors.  It  has  an  on-board  processor  that 
will  output  ASCII  information  in  the  same  format  as  the  Crossbow  IMU,  which  it  replaced. 

•  HipBox:  RIS  Corp  developed  a  multi-channel,  asynchronous  data  logging  unit  called  the 
Handheld  Interface  Pod,  or  HipBox.  The  HipBox  logs  the  serial  output  stream  of  the  G-858, 
EM61,  gyro,  RTS  and  GPS,  stamps  each  recorded  record  with  a  time-tag,  and  produces  a 
standard,  concatenated  ASCII  serial  output  stream.  The  HipBox  output  stream  is  logged  on  a 
PC  tablet  for  real-time  data  merging  and  data  visualization. 

•  RTS:  The  Robotic  Total  Station  is  Leica  model  TSP1200,  which  replaced  the  TSP1 100. 

The  unit  is  a  laser  tracking  system  that  provides  real-time  3D  positions  at  a  rate  of  up  to  8  HZ 
with  sub-cm  accuracy.  The  survey  instrument,  setup  over  a  known  point  in  the  survey  area, 
tracks  a  prism  attached  to  the  sensor  system.  The  positional  data  is  radioed  to  the  roving 
system  and  fed  into  the  HipBox  as  a  serial  stream. 

•  Pushcart:  G-858  cesium- vapor  sensors  were  deployed  using  a  rigid  fiberglass  pushcart. 

The  EM61  was  deployed  using  two  the  standard  wheel/coil  configuration  provided  by 
Geonics  (Figure  2-1,  left. 

The  RIS  HipBox  was  used  to  integrate  all  asynchronous  serial  sensor  output;  including,  the  (i) 
ASCII  positional  data  from  the  Leica  RTS  1200  at  a  rate  of  between  3  and  8  Hertz,  (ii)  binary 
serial  output  from  the  EM61  sensors,  (iii)  serial  output  atlO  Hz  from  the  G-858  sensors,  (iv) 
ASCII  output  from  the  low  metal  gyro  at  a  rate  of  between  10  and  50  Hz,  and  (v)  ASCII  GPS 
output.  The  Trimble  5700  RTK  GPS  provides  a  1  Hz  solution. 
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The  Leica  robotic  total  station  TSP1200  was  used  as  the  primary  positioning  system.  The  unit 
was  mounted  directly  above  the  center  of  the  EM61  coils.  Two  different  prism  mounting  poles 
were  used.  First,  a  short  pole  was  constructed  that  placed  the  prism  20  cm  above  the  top  EM61 
coil.  The  short-pole  has  the  advantage  of  creating  a  smaller  lever-arm  on  the  gimbaled  EM  coils, 
limiting  sway.  The  short-pole  has  the  disadvantage  of  limiting  line-of-sight  needed  by  the  laser 
at  all  times  due  to  operator  obstruction.  As  such,  the  short-pole  option  was  used  only  when  the 
survey  geometry  allowed  for  “broadside”  surveys,  where  the  back-and-forth  survey  lines  did  not 
place  the  operator  in  an  obstructing  position.  A  long-pole  configuration  was  also  used,  where  the 
prism  height  is  established  above  the  operators  head,  thus  avoiding  line-of-sight  issues.  This 
configuration  is  less  desirable  as  it  exacerbates  pendulum-like  motions. 

2.2  Previous  Testing  of  the  Technology 

We  have  performed  preliminary  tests  of  the  Demonstration  data  analysis  and  classification 
technology  during  algorithm  development.  A  number  of  the  user  graphical  interfaces  and 
dialogue  boxes  are  briefly  described  below.  Each  of  these  capabilities  have  been  tested 
individually  and  combined  prior  to  submitting  the  Demonstration  Plan. 

2.2.1  UX- Analyze 

UX-Analyze  allows  users  to  systematically  identify,  extract,  edit,  and  store  data  around 
individual  anomalies.  It  provides  efficient  data  structures  and  access  for  the  analysis  algorithms, 
stores  the  fitted  parameters,  and  allows  for  multiple  data  types  and  surveys.  This  module  is  the 
interface  between  Oasis  montaj™  and  the  Demonstration  analysis  software  (Figure  2). 
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Figure  2.  Screen  snapshots  showing  UX-Analyze  while  selecting  (or  reviewing)  individual 
anomalies. 


2.2.2  Characterization  Modules 

Characterization  routines  for  magnetic  and  EMI  data  have  been  integrated  with  UX-Analyze 
framework.  These  3-D  routines  include  graphic  displays  and  controls  that  allow  the  user  to 
manually  select  and  filter  the  input  data  for  each  anomaly  (Figure  3).  The  derived  model 
parameters  are  stored  in  a  master  target  database.  The  characterization  modules,  or  inversion 
routines,  were  previously  developed  by  AETC  Incorporated  for  the  MTADS  Data  Analysis 
System  (DAS)  under  funding  from  ESTCP  and  SERDP  (Barrow  and  Nelson,  1998;  ESTCP 
Report  199526).  The  MTADS  DAS  codes  were  prototyped  using  the  Interactive  Data  Language 
(IDL). 

Algorithm  equivalency  tests  verified  that  the  C-based  inversion  routines  embedded  in  Oasis 
montaj™  produce  identical  performances  as  the  original  formulations.  Magnetic  and  EMI  data 


7 


were  synthesized  for  forty-nine  sources  that  have  a  unique  combination  of  inclination  and 
declination  but  constant  moment  and  depth  of  burial.  The  layout  and  noise-free  synthetic  data 
are  shown  in  Figure  4  while  Figure  5  shows  screen  snapshots  of  the  user  interfaces  at  various 
stages  of  the  analysis  process.  To  ensure  that  each  routine  received  the  exact  same  input  for  each 
anomaly,  we  extracted  data  samples  around  each  anomaly  individually  once  and  then  used  the 
extracted  data  subsets  as  input  for  both.  Source  parameters  were  then  calculated  using  the  two 
inversion  routines  and  shown  to  be  equivalent  for  dipole  inclinations  of  less  than  75  degrees 
(Tables  1  and  2). 


Figure  3.  Screen  snapshots  showing  the  user  interface  during  data  inversion.  In  this  figure,  the 
measured  data  is  shown  in  the  upper  left  map,  the  model  parameters  (fitted  results)  are  displayed 
in  the  lower  center  window,  and  the  forward  model  generated  using  the  model  parameters  is 
shown  in  the  upper  right  map. 
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Figure  4.  Top  -  schematic  showing  the  inclination  and  declination  of  the  source  objects.  The 
inclination  and  declination  range  from  0  to  90  degrees  in  15-degree  increments.  Bottom  - 
synthesized  electromagnetic  and  magnetic  data,  left  and  right  respectively.  Note  the  subtle 
changes  in  the  shape  of  the  anomalies  for  the  various  combinations  of  source  dipole  orientation. 
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Figure  5.  Screen  snapshots  showing  from  left  to  right  along  each  row  (i)  the  segmentation  of 
individual  anomalies,  (ii)  the  graphical  interface  used  to  identify  and  select  individual  anomalies, 
and  (iii)  anomaly-specific  comparisons  of  measured  and  modeled  data.  Row  (a)  shows  images 
for  UX-Analyze  EMI  data,  (b)  DAS  EMI,  (c)  TJX- Analyze  magnetic,  and  (d)  DAS  magnetic. 
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Table  1.  Comparison  of  Fitted  Results  using  UX- Analyze  and  the  DAS;  magnetic  data. 


X(m) 

YJw) 

Depth  (m) 

|  Size  (m) 

Inc  (deg)  i 

Deo  (deg) 

|  Coherence 

Average  Difference 

0.00 

0.00 

0.00 

0.00 

0.0 

0.0 

0.00 

Standard  Deviation 

0  00 

0  00 

0.00 

0  00 

0.0 

0.0 

0  00 

Table  2.  Comparison  of  Fitted  Results  using  UX- Analyze  and  the  DAS;  electromagnetic  data. 


X  (m) 

Y{m) 

Depth  (m) 

Dec  (deg) 

Inc  (deg) 

Betel 

Beta2 

Beta3 

Coherence 

Average  Difference 

0.00 

0.00 

0.00 

0.0 

0.0 

0.00 

0.00 

0.00 

0.00 

Standard  Deviation 

0.00 

000 

0.00 

0.0 

00 

0.00 

000 

0.00 

000 

2.2.3  Classification  Modules 


Identifying  objects  as  UXO  or  non-UXO  based  on  estimated  target  features  is  notably  difficult 
due  to  imperfect  and  non-unique  feature  estimates.  The  imperfect  and  non-unique  feature 
estimates  result  largely  from  distortions  in  the  spatial  distribution  of  the  anomaly.  The 
distortions  in  turn  are  commonly  caused  by  (i)  errors  in  the  spatial  registration  of  the  measured 
data  samples,  and/or  (ii)  nearby  geologic  or  environmental  signatures  not  associated  with  the 
source  in  question.  As  a  result,  separating  the  fitted  parameters  into  distinct  and  disparate  classes 
is  non-trivial.  Two  classification  methods  that  have  proved  promising  during  the  past  few  years 
include  the  Generalized  Likelihood  Ratio  Test  and  the  Support  Vector  Machine.  Dr.  Leslie 
Collins,  Duke  University,  and  her  colleagues  have  incorporated  both  classifiers  into  UX- Analyze 
as  part  of  this  program.  The  straightforward  graphical  user  interface  to  these  classifiers  is  shown 
in  Figure  6. 


UXO/Clutter  target  classification 


Target  database  file  | arealfeature ,gdb 

Browse 

Training  database  file  J area3feature .gdb 

Browse 

Classification  method  ^  j 

Classification  mode  [Training  and  Classification 

3 

OK  |  .Cancel  |  Help  | 

UXO/Clutter  target  classification 


Input  target  group  |Emi_ 

3 

Input  training  line  |Emi_ 

zi 

Target  ID  channel  | BBaBBBBBflBS 

zl 

Target  mask  channel  iMask 

zl 

UXO/Clutter  training  status  channel  1  Status 

zi 

Feature  channel  1  |EMI_phim 

zl 

Feature  channel  2  (Not  used  if  blank)  |EMI_mp 

~~ 3 

Feature  channel  3  (Not  used  if  blank)  |EMI_mz 

Feature  channel  4  (Not  used  if  blank)  | 

zl 

Feature  channel  5  (Not  used  if  blank)  | 

zl 

Feature  channel  6  (Not  used  if  blank)  | 

zl 

Feature  channel  7  (Not  used  if  blank)  1 

zl 

Feature  channel  8  (Not  used  if  blank)  | 

zl 

Feature  channel  9  (Not  used  if  blank)  1 

zl 

Feature  channel  10  (Not  used  if  blank)  ( 

zl 

OK  |  Cancel  |  Help  | 


Figure  6.  Screen  snapshots  showing  the  user  dialogue  interfaces  that  call  the  classification 
routines. 


11 


2.2.4  Data  Analysis  Documentation 

UX-Analyze  produces  individualized  anomaly  reports,  one  for  each  anomaly,  to  document  the 
decision  process  for  each  anomaly  (Figure  7).  In  each  plot,  the  measured  data  is  graphically 
displayed  next  to  the  modeled  data.  The  model  parameters  are  listed  in  the  middle  of  each  page, 
and  a  profile  extracted  along  the  transect  that  passes  closest  to  the  fitted  dipoles’  location  is 
located  at  the  bottom.  The  positions  of  individual  measurements  are  superimposed  on  the  maps. 


Essentially,  the  anomaly  plots  graphically  provide  an  intuitive  confidence  measure.  If  the 
measured  and  modeled  data  are  indistinguishable,  the  reviewer  can  have  confidence  that  the 
estimated  source  parameters  are  approximately  correct.  If  the  two  maps  are  do  not  resemble 
each  other,  however,  it  tells  us  that  the  source  in  question  (i)  cannot  be  represented  well  using  a 
point  dipole  source,  (ii)  is  not  isolated,  (iii)  does  not  have  sufficient  signal-to-noise  ratio,  or  (iv) 
was  not  properly  sampled  (spatially  or  temporally).  In  any  case,  if  the  two  maps  are  dissimilar 
the  inverted  model  parameters  are  most  likely  incorrect. 


MTADS  Man  Portable  EM61  -  Anomaly  118 


Input  Data 

Fit  Results 

X(m):  64l4Q9.ee 

Y  (m):  4309665.21 

Depth  -0.74 

Inclination  H"  45.3 

Decllnalon  r*  303.3 

Size  (m):  0.156 

Coherence:  0.9037 

Solid  Angle  (°):  35.1 


Modeled  Date 


MTADS  vehicle  magnetic  data  -  Anomaly  118 


Figure  7.  To  document  the  analysis,  UX-Analyze  generates  a  one-page  summary  for  each 
anomaly.  In  the  anomaly  summaries  shown  above,  the  measured  data  is  shown  in  the  upper  left 
hand  comer,  the  inverted  model  parameters  in  the  middle  left,  the  forward  model  in  the  upper 
rights,  and  a  profile  in  the  lower  left  comer.  This  layout  was  selected  to  provide  insight  into  the 
confidence  of  the  analysis  and  conclusions.  EMI  data  for  the  anomaly  are  shown  in  the  left 
summary,  and  magnetic  data  on  the  right. 
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2.3  Factors  Affecting  Cost  and  Performance 


Costs  to  collect  the  data  required  by  advanced  processing  are  greater  than  that  required  for  a 
detection  survey.  Higher  acquisition  cost  result  from  data  density,  spatial  registration,  and  signal 
to  noise  requirements  that  are  necessary  to  support  discrimination  processing.  In  addition  to 
higher  field  costs,  more  analysts’  time  is  required  for  discrimination  efforts  compared  to 
detection  only  objectives. 


2.4  Advantages  and  Limitations  of  the  Technology 

This  demonstration  technology  uses  spatially  referenced  geophysical  data  to  estimate  target 
parameters  for  each  anomaly.  The  approach  assumes  a  dipolar  source.  This  has  an  inherent 
advantage  over  add  hoc  analysis  methods  that  are  commonly  used.  Due  to  a  lack  of  analysis 
routines  available,  many  contractors  make  UXO  and  non-UXO  declarations  based  on  anomaly 
amplitude,  half  width,  spatial  footprint,  or  overall  Took’.  These  characterization  methods  are 
sensitive  to  the  targets’  orientation  and  depth  of  burial  (or  distance  from  the  sensor).  The 
methodology  demonstrated  here  separates  the  measured  signatures  into  that  which  is  inherent  to 
the  target,  and  that  which  is  related  to  the  geometry  of  the  problem  (such  as  distance  to  sensor 
and  orientation). 

The  primary  advantage,  therefore,  is  the  potential  for  discriminating  between  UXO  and  non 
UXO-like  objects  based  upon  geophysical  survey  data.  This  is  in  contrast  to  simply  identifying 
the  location  of  anomalies  from  the  geophysical  survey  data.  Results  from  past  demonstration 
have  shown  that  that  some  discrimination  is  possible  using  magnetic  and  electromagnetic  data 
(Robitaille  et  al.,  1999).  Magnetic  discrimination  is  based  primarily  on  the  apparent  fitted  dipole 
moment  (or  apparent  size  if  scaled  correctly).  Using  EMI  data,  increased  discrimination 
performance  can  sometimes  be  achieved  by  utilizing  estimated  shape  information.  If  successful 
discrimination  capabilities  can  be  achieved,  significant  excavation  savings  can  be  realized  by 
leaving  the  non-hazard  clutter  items  unearthed. 

This  is  not  to  say,  however,  that  the  technology  being  demonstrated  will  solve  the  UXO 
characterization  and  classification  problems.  Even  with  optimal  data  quality,  the  estimated  fit 
parameters  cannot  always  be  separated  into  distinct,  non-overlapping  classes  of  UXO  and  non 
UXO-like  objects.  In  addition,  none  of  the  fit  parameters  are  actually  unique  to  UXO  items. 
Clutter  items  that  physically  resemble  UXO  will  probably  be  misclassified.  Additionally,  if  the 
data  quality  is  not  optimal  the  fitted  parameters  cannot  be  trusted. 
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3.  Demonstration  Design 


3.1  Performance  Objectives 


Type  of 
Performance 
Objective 

Primary 

Performance 

Criteria 

Expected 

Performance 

(metric) 

Actual  Performance 
(all  targets) 

Quantitative 

Analysis  time 

Characterize: 

<5  min/anomaly 
Document: 

<30  sec/anomaly 

Characterize: 

3min  per  anomaly 
Document: 

<15sec  per  anomaly 

%  UXO  Detected 

>90 

84 

%  UXO  Classified 

>90 

0.74 

False  Negative 

0 

0.15 

Qualitative 

Ease  of  use 

General  Observations 

See  comments 
section  4.3.4 

Robustness 

General  Observations 

See  comments 
section  4.3.4 

We  ranked  the  classifier  outputs  using  a  scheme  of  1,  2,  and  3,  where  a  rank  of  1  indicates  high 
confidence  UXO,  rank  3  is  used  for  high  confidence  clutter,  and  rank  2  is  everything  else.  The 
percentage  UXO  classified  is  defined  as  the  Pd  for  rank  1  anomalies.  The  false  negative  metric 
is  defined  as  those  anomalies  that  are  classified  high  confidence  clutter,  but  were  in  fact  UXO.  It 
corresponds  to  (1-Pd)  of  the  rank  3  anomalies. 

3.2  Selecting  the  Test  Site 

The  demonstration  site  was  selected  because  (i)  there  is  a  real  and  ongoing  need  to  remediate  the 
site  in  order  to  sell  and  develop  the  land,  (ii)  DuPont  and  AETC  Incorporated  have  demonstrated 
past  success  utilizing  cued  data  acquisition  approaches,  and  (iii)  advancements  in  laser  ranging 
systems  led  us  to  believe  that  discrimination  quality  data  (spatial  registration  precision  at  the 
centimeter  level)  could  be  acquired  in  lightly  forested  areas.  The  overarching  financial 
motivation  was  to  save  monies  by  basing  the  discrimination  decision  on  field  survey  data  instead 
of  cued  data. 

Two  site  visits  were  conducted  prior  to  selecting  the  site  and  requesting  approval  from  the 
ESTCP  Program  Office.  Present  at  the  initial  meeting  were  R.  Mehl,  Sky  Research,  D. 
Keiswetter,  AETC  Incorporated,  and  B.  Ambrose,  DuPont.  At  the  initial  meeting,  B.  Ambrose 
identified  a  portion  of  DuPont’s  site  for  the  demonstration  so  that  Sky  Research  could  to  assess 
the  feasibility  of  successfully  deploying  their  hardware  at  this  locale.  Present  at  the  second  site 
walkover,  which  was  conducted  to  confirm  the  suitability  of  the  five-acre  down  select,  were  J. 
Foley,  Sky  Research,  T.  Bell,  AETC  Incorporated,  and  B.  Ambrose. 


14 


3.3  Test  Site  History/Characteristics 


Lake  Success  Business  Park  (LSBP)  is  a  422-acre  underdeveloped  industrial  property  located  in 
Bridgeport  and  Stratford,  Connecticut  (Figure  8).  Remington  Arms  Company,  Inc.  has  owned 
the  site  the  late  1800’s.  Remington  manufactured,  tested  and  stored  small-caliber  and  other 
ammunition  at  the  site.  After  the  Vietnam  War,  site  operations  scaled  back  and  the  facility 
eventually  ceased  production  in  1989.  E.  I.  duPont  de  Nemours  &  Co.  (DuPont)  divested  all 
operating  Remington  Arms  facilities  in  1993,  but  retained  ownership  of  the  closed 
Bridgeport/Stratford  site.  The  Remington  Arms  name  was  transferred  with  the  sale  of  the 
operating  facilities.  Remington  Arms  was  renamed  Sporting  Goods  Properties,  Inc.  (SGPI)  and 
remains  a  subsidiary  of  DuPont.  The  site,  formerly  know  as  Remington  Park  or  Remington 
Woods,  was  renamed  Lake  Success  Business  Park.  Since  1989,  177  buildings  containing 
asbestos  and  residual  explosive  materials  have  been  demolished.  The  site  is  presently  engaged  in 
an  aggressive  remediation  program.  DuPont  has  received  the  approvals  of  the  regulatory 
agencies  to  proceed  with  redevelopment  of  this  parcel  (Figure  9).  During  remediation  of  areas  of 
environmental  concern  at  other  locations  on  LSBP  both  small  and  large  caliber  munitions  were 
found  in  places  where  they  were  not  expected.  As  an  added  precaution  prior  to  releasing  this 
portion  of  the  site  for  redevelopment  DuPont  wants  to  conduct  UXO  survey  utilizing  current 
state  of  the  art  detection  and  mapping  technologies  and  field  verification  to  determine  if  UXOs 
are  present. 

LSBP  is  an  interim  status  Resource  Conservation  and  Recovery  Act  (RCRA)  facility.  On 
August  21,  1990,  Remington  Arms  Company,  Inc.  entered  into  an  Administrative  Consent  Order 
(ACO)  with  USEPA.  The  ACO  has  been  modified  twice.  The  first  modification  (1994) 
authorized  the  construction  of  a  Corrective  Action  Management  Unit,  which  is  a  facility 
designed  to  accumulate  and  treat  contaminated  soils.  The  second  modification  (1997)  approved 
the  implementation  or  Remedy  I  (excavation  and  treatment  of  upland  soils).  A  Phase  I  RCRA 
Facility  Investigation  was  performed  in  1989-91  and  approved  by  EPA  in  1992.  Fifty-one  areas 
of  environmental  concern  were  identified  and  investigated.  The  primary  contaminant  is  lead. 
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Figure  8.  DuPont’s  Lake  Success  Business  Park  is  located  between  Stratford  and  Bridgeport, 
CT. 
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Figure  9.  Lake  Success  Business  Park,  the  open  forest  region  in  the  center  of  this  satellite  image, 
is  located  in  an  underdeveloped  area  of  Bridgeport  and  Stratford,  Connecticut.  The  development 
disparity  with  regards  to  the  surrounding  area  is  evident  in  this  satellite  image. 

3.4  Present  Operations 

During  2000  and  2001,  DuPont  hired  a  commercial  contractor  to  survey  seventy  acres  of  the 
LSBP  site.  This  acreage  was  selected  because  of  its  priority  in  the  transition  process.  As  a  result 
of  the  survey,  over  2,100  anomalies  were  identified  (Figure  10).  The  data  quality  and  density  did 
not  support  advanced  analysis.  An  intrusive  investigation  of  approximately  300  anomalies  in  a 
four-acre  portion  of  the  site  yielded  a  plethora  of  wire  items  and  metallic  debris,  a  significant 
number  of  ‘no  finds’,  and  no  ordnance  items.  Aside  from  the  excavation  within  four-acre  parcel, 
no  subsequent  action  has  occurred  within  the  original  boundaries  of  the  70-acre  investigation 
area.  The  five-acre  parcel  associated  with  this  Demonstration  is  within  the  original  70-acre 
survey. 

In  2002  DuPont  contracted  AETC  Incorporated  to  conduct  a  digital  geophysical  mapping  survey 
to  identify  potential  UXOs  over  an  18-acre  parcel  in  the  southwestern  comer  of  LSBP.  The  18- 
acre  parcel  was  not  included  in  the  prior  70-acre  survey.  This  acreage  was  also  selected  in 
response  to  transition  plans  and  schedules.  DuPont’s  objective  was  to  render  clean  the  18-acre 
parcel  in  Bridgeport.  Prior  to  surveying  the  18  acres,  a  test  bed  containing  a  number  of  37mm, 
47mm  and  75mm  surrogates  buried  at  various  depths  and  orientations  as  well  as  various  clutter 
items  was  established  and  surveyed  in  August  2002.  The  test  bed  was  surveyed  with  a  GEM-3, 
an  EM61  Mark  2  (EM61)  and  a  SmartMag  magnetometer.  Based  on  the  results  of  the  test  bed 
surveys  the  EM61  MKII  was  selected  for  the  18 -acre  parcel. 

In  March  2003,  AETC  reported  approximately  1800  anomalies  in  the  18-acre  parcel.  Each 
anomaly  was  categorized  as  (1)  possible  UXO,  (2)  uncertain  /  more  information  needed,  or  (3) 
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high  confidence  not  UXO.  This  effort  essentially  focused  on  reducing  costs  by  identifying  those 
anomalies  that  were  not  UXO;  852  of  1,546  contacts  were  ruled  out  based  on  the  analysis  of 
EM61  MKII  survey  data.  Each  of  the  remaining  694  anomalies  was  relocated  and  surveyed 
using  an  EM61-HH  sensor  in  a  cued  mode.  The  priorities  were  then  reassigned  based  on  EM61- 
HH  processing  for  each  of  the  694  anomalies.  This  process  eliminated  584  (84%)  of  the  694 
potential  UXO  contacts  using  feature-based  analysis  of  the  EM61-HH  cued  data.  The  remaining 
1 10  anomalies  were  targeted  for  intrusively  investigated.  In  the  fall  of  2003  all  but  40  of  the 
anomalies  were  investigated.  No  UXO  were  found.  The  cost  savings  on  excavation  and  blast 
containment  is  estimated  at  $1  million  (Ambrose,  2004).  The  expenditures  for  the  EM61-HH 
data  collection  and  discrimination  processing  were  $45k  and  $3  5k  respectively. 
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Figure  10.  Left:  Map  showing  the  70-acre  grid  boundaries  and  anomaly  density  (black  circles) 
from  the  commercial  contractor  survey.  The  five-acre  subset  that  is  the  area  of  interest  for  this 
demonstration  is  colored.  Right:  EM61  map  of  the  five-acre  area  selected  for  this 
demonstration. 

3.5  Pre-Demonstration  Testing  and  Analysis 

Calibration  data  were  acquired  for  a  variety  of  ordnance  objects  prior  to  the  start  of  field  work. 
Anecdotal  evidence  indicated  that  munitions  including  37mm,  40mm,  57mm,  60mm,  105mm 
and  2.75in  mortars  may  be  present.  Because  no  recovered  inert  ordnance  at  the  site  existed,  we 
utilized  inert  UXOs  provided  by  the  Army  Environmental  Center.  EM61  Mkll  data  were 
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acquired  in  controlled  tests  using  the  same  sensor  that  would  be  used  for  production  work.  Data 
were  taken  for  each  ordnance  item  in  six  orientations.  The  orientations  were  nose-up,  nose- 
down,  horizontal  in  X,  horizontal  in  Y,  45  degree  nose-up,  and  45  degree  nose-down.  The 
sensor  coils  were  horizontal  as  shown  below. 


Figure  1 1  shows  the  recovered  longitudinal  versus  transverse  response  coefficients.  There  are 
six  data  point  per  ordnance  type  (one  for  each  sensor-UXO  configuration;  nose-up,  nose-down, 
horizontal  in  X,  horizontal  in  Y,  45  degree  nose-up,  and  45  degree  nose-down).  The  contours 
encircle  an  area  that  is  two  times  the  standard  deviation  about  the  mean  for  each  ordnance  type. 


Beta-T  vs  Beta-L  for  Time  gate  2 


Figure  11.  Feature  plot  showing  longitudinal  versus  transverse  response  coefficients  for 
emplaced  UXO  types.  The  six  data  points  per  target  represent  distinct  coil-UXO  geometries. 
The  contours  are  centered  about  the  mean  of  the  measurements  and  extend  two  standard 
deviations  from  the  mean. 

3.6  Testing  and  Evaluation  Plan 

3.6.1  Demonstration  Set-up  and  Start-up 

Inert  ordnance  from  the  Aberdeen  Test  Center  (ATC)  APG  MD  was  seeded  within  the  survey 
area  to  ensure  the  presence  of  UXO.  ATC  degaussed  the  ordnance  and  transferred  it  to 
Connecticut  for  emplacement  on  the  site.  Mr.  Brian  Ambrose  and  Mr.  Jeffrey  Fairbanks, 
HydroGeoLogic,  Inc.  emplaced  the  ordnance.  The  ground  truth  for  the  seeded  targets  was  held 
until  after  the  completion  of  the  survey  -  it  was  not  available  to  any  data  analyst  associated  with 
this  project  until  the  completion  of  their  assigned  analyses.  The  location  of  the  emplaced 
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ordnance  is  shown  in  Figure  12.  The  emplaced  ordnance  types  included  37mm  projectiles, 
40mm  grenades,  40mm  projectiles,  57mm  projectiles,  60mm  mortars  105mm  projectiles,  and 
2.75inch  rockets.  The  ordnance  was  buried  at  depths  shallower  than  eleven  times  their  diameter. 

DuPont  paid  for  and  oversaw  brush  clearing  activities  during  May  2004.  Photographs  of  the  site 
before  and  after  clearance  activities  are  shown  in  Figure  13. 

3.6.2  Period  of  Operation 

Sky  Research  acquired  EMI  and  magnetic  data  during  June  2004.  Sky  Research  mobilized  the 
field  crew  to  the  site  on  June  1  and  demobilized  Tuesday  June  22. 

3.6.3  Area  Characterized 

Sky  Research  surveyed  an  area  that  was  0.83  hectare  (2.0  acres)  in  size  using  the  EM  sensor 
system  and  0.95  hectare  (2.4  acres)  using  the  magnetic  system  (Figure  14). 

Magnetic  data  were  acquired  using  two  different  platforms,  a  pushcart  and  litter  configuration. 
Photographs  of  the  two  systems  are  shown  in  Figures  15  while  the  coverage  is  shown  in  Figure 
16.  Because  of  its  design,  the  pushcart  platform  had  difficulty  operating  in  grasses,  short  brush 
stubble,  and  crossing  tire  ruts  or  surface  debris.  A  litter  configuration  was  constructed  and 
utilized  June  9. 
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Figure  13.  Site  photographs  -  the  top  two  photographs  were  taken  prior  to  clearing  the  brush. 
The  bottom  two  photographs  were  taken  after  brush  clearing  was  completed  in  June  2004. 
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Figure  14.  Coverage  map  showing  the  areas  covered  by  EMI  only,  mag  only,  or  both. 
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Figure  15.  Photographs  of  the  two  magnetometer  platforms  utilized  during  data  acquisition.  The 
pushcart  system  is  shown  in  the  left  photo  and  the  litter  configuration  on  the  right. 


Figure  16.  Site  map  showing  magnetic  coverage  using  the  pushcart  system  (black  symbols)  and 
the  litter  configuration  (blue). 
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In  addition  to  the  production  survey  data,  EM61  MKII  data  were  acquired  in  a  grid  over  twenty 
emplaced  targets  and  twenty  anomalies  identified  in  the  original  Foster- Wheeler  survey  data 
(Figure  17).  Collecting  data  in  a  static  grid  configuration  minimizes  errors  in  positioning  that 
area  associated  with  survey  motion.  The  cued  data  were  collected  on  a  6x5  point  rectangle  grid 
(nodes  separated  by  20.3cm  [8inches]  and  40.6cm  [16inches]  respectively  resulting  in  a 
101.5x162.4cm  [3  %  -  x  5%-foot]  area).  The  grid  was  elevated  20  cm  above  the  ground,  and  the 
EM61  was  positioned  directly  on  the  grid  nodes,  without  wheels.  Sensor  locations  on  the  grid 
were  controlled  by  lining  up  cross  hairs  on  the  EM61  bottom  coil  with  grid  line  intersections. 
The  choice  of  grid  layout  and  elevation  was  based  on  results  of  simulations  of  data  inversion 
sensitivity. 


Figure  17.  Photographs  taken  during  acquisition  of  EM61  Mkll  data  using  a  template.  The  cued 
approach  minimizes  motion  noise  and  errors  in  the  spatial  registration  of  the  geophysical 
readings. 


3.6.4  Demobilization 

At  the  conclusion  of  the  survey,  all  Sky  Research  equipment  was  packed  and  transported  to  a 
commercial  shipping  company. 

3.6.5  Data  Processing  Steps 

As  mentioned  above  in  section  2.1.2,  all  sensor  and  spatial  registration  data  were  read  into  the 
HipBox  as  asynchronous  serial  records.  When  received,  the  HipBox  stamps  each  incoming 
record  with  the  HipBox  time,  reformats  each  data  record,  and  concatenates  the  various  data  into 
a  multiplexed  output  stream.  In  terms  of  serial  input  data,  the  HipBox  acts  as  a  data 
accumulator,  reformatting  and  time-stamping  all  input  data.  Once  all  data  are  reformatted  and 
brought  to  a  common  time-base,  the  data  are  logged  on  a  tablet  PC  using  a  customized  data 
logging  program;  RisLogger.  This  program  provides  several  data  manipulation,  merging,  and 
visualization  functions.  One  essential  task  in  the  data  merging  is  the  establishment  of  the 
position  of  each  sensor  reading  (sensor  value  and  time)  based  on  the  data  provided  by  the  RTS 
(prism  position  and  time).  This  is  done  for  G-858,  EM61  and  gyro  data. 
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4.  Performance  Assessment 


4.1  Performance  Criteria 


Table  3.  Performance  Criteria 


Performance 

Criterion 

Description 

Primary  or 
Secondary 

Analysis  Time 

Times  required  making  a  decision  for  each  anomaly  and 
prepare  required  documentation  supporting  the  decision. 
Analysis  time  will  be  logged  into  two  categories:  A) 
identifying,  characterizing,  and  classifying  individual 
anomalies,  and  B)  preparing  anomaly  specific  images  and 
dig  sheet  information. 

Primary 

%  UXO  Detected 

(#  of  detections  at  emplaced  ordnance  locations)/(#  of 
emplaced  ordnance) 

Primary 

%  UXO  Classified 

(#  of  anomalies  that  are  classified  as  a  priority  l)/(#  of 
actual  ordnance  found) 

Primary 

#  False  Negatives 

Anomaly  classified  as  high  confidence  clutter  (priority  3) 
that  is  actually  a  UXO 

Primary 

Ease  of  Use 

Describe  the  processing  flow  and  the  anticipated  skill  level 
required.  Although  the  data  analysts  working  on  this 
demonstration  are  not  novices  with  regard  to  UX- Analyze, 
the  goal  of  the  program  is  that  data  processors  experienced 
with  Oasis  montaj™  could  utilize  the  software  with  the  on¬ 
line  help  and  without  hands-on  training. 

Primary 

Robustness 

No  major  bugs  that  artificially  limit  the  analysts’  ability  to 
conduct  analysis. 

Primary 

4.2  Performance  Confirmation  Methods 

The  Sky  Research  magnetic  and  EMI  systems  surveyed  portions  of  the  site  in  which  the  ESTCP 
Program  Office  arranged  for  a  variety  of  inert  ordnance  to  be  seeded.  After  analysis  of  the  data, 
we  submitted  target  lists  to  the  Program  Office.  Using  the  known  positions  of  these  emplaced 
targets,  the  detection  and  discrimination  performance  was  calculated. 

4.3  Data  Analysis,  Interpretation,  and  Evaluation 

4.3.1  Detection 

The  emplaced  targets  can  be  utilized  to  assess  the  detection  performance  because  we  know  their 
number  and  locations.  The  ESTCP  Program  Office  arranged  for  102  inert  UXO  items  to  be 
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emplaced  within  the  5-acre  area.  The  emplaced  items  included  7  105mm  projectiles,  17  2.75- 
inch  warheads,  17  60mm  projectiles,  16  57mm  projectiles,  21  40mm  grenades,  21  40mm 
projectiles,  and  3  37mm  projectiles.  In  addition,  one  of  each  ordnance  type  was  buried  within 
the  survey  grid  to  serve  as  calibration  targets.  The  locations  and  details  of  the  seven  calibration 
targets  were  provided  to  Sky  Research  personnel  for  system  performance  tests. 

Not  all  of  the  emplaced  targets  were  surveyed  by  Sky’s  EMI  sensor  or  by  their  magnetic 
systems.  Figure  18  is  color-coded  to  show  which  instrument  surveyed  each  of  the  emplaced 
targets.  Of  the  102  emplaced  targets,  only  69  targets  were  surveyed  by  Sky’s  EMI  system  and 
60  by  their  magnetic  systems  (Figures  19  and  20). 

Of  the  69  emplaced  targets  surveyed  with  the  EMI  sensor,  58  were  selected  as  potential  objects, 
resulting  in  84%  detection  rate.  Of  the  eleven  emplaced  items  that  were  not  picked  as  targets  in 
the  EMI  data,  three  had  data  problems  and  eight  were  at  or  below  background  noise  levels.  The 
majority  of  the  missed  targets  were  40mm  projectiles  or  grenades.  Excluding  the  40mm 
projectiles  and  grenades,  one  105mm  projectile  and  two  57mm  mortars  were  not  detected. 
Figure  21  shows  the  105mm  and  one  of  the  57mm  projectiles  that  were  not  picked  at  the 
emplaced  targets’  location.  Table  4  summarizes  the  detection  results  by  ordnance  type  and 
sensor. 

Of  the  50  emplaced,  ferrous  targets  surveyed  with  the  magnetic  sensors,  33  were  selected  as 
potential  targets,  resulting  in  66%  UXO  detection  rating.  Half  of  the  missed  targets  were  40mm 
or  57mm  projectiles.  Figure  22  provides  examples  of  emplaced  targets  for  which  an  anomaly 
was  not  detected  and  declared. 


Table  4.  Detection  Performance 

?y  Ordnance  Type  and  Sensor  Modality 

Item 

Total 

EM 

Surveyed 

Detected 

Mag 

Surveyed  Detected 

37mm  Projectile 

3 

2 

2 

1 

0 

40mm  Grenade** 

21 

15 

8 

10 

1 

40mm  Projectile 

21 

12 

11 

8 

4 

57mm  Projectile 

17 

9 

7 

13 

6 

60mm  Mortar 

18 

11 

11 

11 

8 

2.75"  Rocket 

17 

14 

14 

11 

10 

105mm  Projectile 

7 

6 

5 

6 

5 

non-ferrous 
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Figure  18.  Color-coded  map  of  the  emplaced  inert  UXO  items  showing  which  were  surveyed  by 
Sky’s  magnetic  system,  EMI  system,  or  both. 
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Figure  19.  False  color  map  showing  the  EMI  data  coverage  overlain  by  locations  of  the  emplaced 
targets  and  those  declared  detected  during  analysis. 
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Figure  20.  False  color  map  showing  the  magnetic  data  coverage  overlain  by  locations  of  the 
emplaced  targets  and  those  declared  detected  during  analysis. 
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Figure  21.  Anomaly  plots  of  two  emplaced  targets  that  were  not  picked  as  targets  in  the  EMI  data.  Left  Anomaly  Plot  -  The 
location  of  a  105mm  projectile  is  identified  by  the  cross  in  the  upper  left  map,  but  no  signature  is  observed.  Right  Anomaly 
Plot  -  The  location  of  a  57mm  projectile  is  identified  by  the  “X”  in  the  upper  left  map.  The  ‘Comments’  field,  which  is 
located  just  above  the  profile  in  the  two  plots,  reports  the  UXO  type  from  the  ground  truth  information. 
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Figure  22.  Anomaly  plots  of  two  emplaced  targets  that  were  not  picked  as  targets  in  the  magnetic  data.  Left  Anomaly  Plot  - 
The  location  of  a  40mm  projectile  is  identified  by  the  cross  in  the  upper  left  map.  The  spatial  sampling  pattern  shown  in  the 
plot  presumably  resulted  from  trying  to  push  and  twist  the  pushcart  through  the  grasses  and  cut  debris.  Right  Anomaly  Plot- 
The  signature  of  a  57mm  grenade,  identified  by  the  “X”  in  the  upper  left  map,  is  within  the  background  noise.  The 
‘Comments’  field,  which  is  located  just  above  the  profile  in  the  two  plots,  reports  the  UXO  type  from  the  ground  truth 
information. 
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4.3.2  Discrimination 


4.3.2. 1  Comparison  of  key  Decision  Metrics;  Cued  vs.  Survey  Data 

Because  emplaced  UXO  are  symmetric  and  their  shape  is  known  and  consistent,  we  can  use 
them  to  evaluate  the  fitted  parameters  with  respect  to  modeling  errors  and  shape  information. 
There  are  two  fundamentally  important  measures.  The  first  has  to  do  with  the  ability  of  the 
inversion  routine  to  reproduce  the  measured  signal.  In  other  words,  how  well  does  the 
modeled  data  match  that  which  is  measured?  If  the  match  is  poor,  the  significance  of  the 
inverted  source  parameters  is  diminished.  For  this  measure,  we  show  the  dipole  fit  error, 
defined  as  the  square  root  of  1  minus  the  squared  correlation  coefficient  between  the 
measured  and  modeled  data.  The  second  measure  relates  to  information  regarding  the 
estimated  shape  of  the  source  object.  For  this  measure,  we  utilize  the  absolute  values  of  the 
three  magnetic  polarizability  response  coefficients. 

A  plot  of  fit  errors  versus  maximum  signal  is  shown  in  figure  23.  In  this  figure,  the  red 
diamonds  represent  fitted  parameters  from  the  cued  data  and  blue  triangles  identify  survey 
data.  The  data  shown  here  include  only  those  emplaced  objects  that  were  surveyed  using 
both  acquisition  approaches.  As  is  clearly  evident,  the  cued  data  possess  higher  amplitudes 
and  significantly  smaller  fit  errors  than  Sky’s  production  survey. 
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Figure  23.  Scatter  plot  of  the  maximum  signal  versus  dipole  fit  error.  The  data  in  this  figure 
are  restricted  to  emplaced  UXO  objects  that  were  surveyed  by  Sky’s  production  EM61 
survey  and  the  cued  deployment. 

As  mentioned  above,  shape  information  can  be  a  powerful  discriminate  metric  if  the  aspect 
ratios  and/or  axial  symmetry  values  of  the  UXO  and  clutter  are  separable.  The  three  time- 
dependent  dipole  response  coefficients  relate  to  the  principle  axes  of  the  target.  The  ratio  of 
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the  largest  response  coefficient  to  the  mean  of  the  remaining  two,  therefore,  provides  an 
estimate  of  the  aspect  ratio.  Similarly,  the  two  transverse  response  coefficients  provide 
information  about  the  axial  symmetry.  If  the  transverse  coefficients  are  equal,  the  object  can 
be  thought  to  be  axially  symmetric. 

Inverted  shape  measures  for  the  emplaced  targets  are  shown  graphically  in  figure  24  and 
detailed  in  Table  5.  In  this  figure,  a  log-log  plot  is  used  to  visualize  the  feature  space  because 
of  the  tremendous  range  of  values.  The  color-coded  contours  are  derived  from  the  test  stand 
data  and  the  symbols  represent  features  derived  from  cued  or  survey  data  as  indicated  (each 
symbol  represents  fit  results  for  one  anomaly).  The  contours  are  centered  about  the  mean 
measurement  and  extend  two  standard  deviations  for  each  ordnance  type.  Symbols  on  the  y- 
axis  are  plotted  at  the  average  of  the  two  smaller  coefficients  and  the  vertical  line  identifies 
the  range.  Plotted  in  this  way,  small  vertical  lines  indicate  axial  symmetry. 
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Figure  24.  Magnetic  polarizability  coefficients  for  emplaced  targets,  EM61  Mkll  data,  gate  2. 
For  the  transverse  coefficients  plotted  on  the  y-axis,  symbols  identify  the  mean  of  the  two 
smaller  magnetic  response  coefficients  and  lines  show  the  range. 
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Table  5  presents  details  regarding  the  derived  shape  indicators.  Here,  we  list  the  dipole  fit 
error,  estimated  aspect  ratio,  and  an  asymmetry  metric  for  some  of  the  emplaced  UXO 
targets.  The  aspect  ratio  is  derived  by  dividing  the  largest  inverted  response  coefficient  with 
the  average  of  the  smaller  two. 

As  a  rule,  the  estimated  shape  parameters  from  the  cued  data  reproduced  the  measured 
signatures  better  than  the  survey  data.  The  mean  fit  error,  which  is  defined  as  the  square  root 
of  1  minus  the  squared  correlation  coefficient  between  the  measured  and  modeled  data,  was 
0.066  for  the  cued  compared  to  0.321  for  the  survey  data. 

The  asymmetry  metric  is  defined  here  as  the  difference  between  the  two  smaller  response 
coefficients  divided  by  the  larger  of  the  two.  Expressed  this  way,  the  asymmetry  metric 
ranges  from  0  to  1 ,  were  0  indicates  that  the  two  smaller  response  coefficients  are  equal  and 
therefore  axially  symmetric  and  a  value  of  1  essentially  indicates  that  the  smaller  response 
coefficient  is  equal  to  zero  (to  four  decimal  places).  Remember  that  because  the  items 
included  in  this  comparison  are  inert  UXO,  they  all  possess  axial  symmetry  and  ideally 
would  result  in  asymmetry  values  of  0.  As  shown  below,  however,  asymmetry  values  from 
the  cued  data  show  some  scatter  and  a  mean  value  of  25%.  Asymmetry  values  from  Sky’s 
survey  data  possess  considerably  more  variations  and  a  mean  value  of  more  than  60%.  Note 
that  these  asymmetry  measures  do  not  include  the  40mm  grenade  because  it  is  non-ferrous. 
Unlike  ferrous  objects,  in  which  B1  is  the  largest  with  smaller  and  equal  B2  and  B3  values, 
non-ferrous  objects  have  roughly  equal  B1  and  B2  values  and  a  smaller  B3  value.  Because 
of  this,  the  asymmetry  metric  as  defined  here  doesn’t  apply  to  the  40mm  grenade. 


Table  5.  Fit  Results  for  Emplaced  Targets;  Cued  vs.  Sky  Survey  Data 


Cued  Mode  Fit  Results 

Survey  Mode  Fit  Results 

ID 

Object 

Burial 
Depth  (m) 
(mid  pt) 

Fitted 
Depth  (m) 

Fit  Error 
sqrt[1-(rA2)] 

Aspect  Ratio 
[B1/avg(B2,B3)j 

Asymmetry 

[(B2-B3)/B2] 

Fitted 
Depth  (m) 

Fit  Error 
sqrt[1-(rA2)] 

Aspect  Ratio 
[B1/avg(B2,B3)] 

Asymmetry 

[(B2-B3)/B2] 

8 

37mm 

0.07 

0.12 

0.049 

4.0 

44% 

0.01 

0.469 

2.8 

55% 

88 

40mm  Grenade* 

0.09 

0.03 

0.128 

1.7 

60% 

0.03 

0.459 

3.4 

74% 

94 

40mm  Projectile 

0.12 

0.14 

0.047 

2.8 

8% 

0.26 

0.539 

30.5 

100% 

100 

57mm 

0.08 

0.05 

0.024 

1.6 

7% 

0.09 

0.180 

2.0 

26% 

103 

57mm 

0.22 

0.19 

0.132 

1.9 

36% 

0.08 

0.286 

1.7 

45% 

43 

60mm 

0.32 

0.30 

0.132 

4.0 

37% 

0.32 

0.593 

1.5 

47% 

107 

60mm 

0.10 

0.10 

0.020 

3.4 

6% 

0.14 

0.176 

1.6 

49% 

108 

60mm 

0.14 

0.15 

0.037 

3.5 

20% 

0.30 

0.286 

4.9 

99% 

110 

60mm 

0.22 

0.31 

0.107 

3.4 

9% 

0.22 

0.686 

2.4 

77% 

9 

105mm 

0.18 

0.25 

0.033 

2.0 

13% 

0.28 

0.164 

1.8 

36% 

10 

105mm 

0.23 

0.32 

0.022 

2.0 

11% 

0.30 

0.126 

1.6 

16% 

11 

105mm 

0.49 

0.57 

0.020 

2.1 

5% 

0.41 

0.103 

1.5 

29% 

47 

105mm 

0.57 

0.77 

0.077 

2.1 

38% 

0.59 

0.328 

2.6 

77% 

80 

2.75in 

0.17 

0.23 

0.041 

10.4 

30% 

0.22 

0.254 

6.8 

55% 

82 

2.75in 

0.31 

0.35 

0.026 

11.7 

52% 

0.28 

0.238 

8.2 

80% 

84 

2.75in 

0.44 

0.44 

0.114 

14.8 

40% 

0.53 

0.301 

8.6 

100% 

85 

2.75in 

0.64 

0.72 

0.116 

8.7 

45% 

0.55 

0.277 

2.4 

75% 

MEAN  0.066  25%  0.321  60% 

STD  DEV  0.045  17%  0.170  27% 


*  non-ferrous;  not  included  in  the  mean  and  standard  deviation  asymmetry  measures 
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As  mentioned  earlier,  cued  measurements  were  also  taken  for  some  native,  non-emplaced 
targets.  These  targets  were  identified  using  data  from  an  EM61  reconnaissance  survey 
conducted  by  Foster  Wheeler  in  2000.  The  details  of  the  source  object  were  not  known  while 
selecting  targets.  Excavation  results  reveal  that  all  of  the  native  targets  were  clutter,  some  of 
which  were  axially  symmetric. 

Relevant  data  quality  and  shape  metrics  are  presented  in  Table  6  for  objects  that  were 
photographed,  relatively  isolated,  and  possessed  acceptable  signal  to  noise  ratios. 
Representative  photographs  are  shown  in  Figure  25.  Once  again,  we  look  at  data-model  fit 
error,  aspect  ratio,  and  an  asymmetry  metric  derived  from  the  cued  data  as  well  as 
corresponding  values  derived  from  Sky’s  production  data. 

First,  note  that  the  fit  error  between  the  measured  and  modeled  data  for  the  cued  approach  is 
significantly  smaller  than  the  survey  mode  data.  For  these  targets,  the  mean  fit  error  for  the 
cued  measurements  was  0.095  while  the  survey  mode  data  had  a  mean  fit  error  of  0.3 14. 
Second,  note  that  with  the  exception  of  ID  #14  (which  was  actually  two  moderately  sized 
metallic  fragments),  the  asymmetry  values  for  those  targets  without  axial  symmetry  (ID’s  78 
and  60)  were  greater  than  50%  for  the  cued  data  while  those  with  axial  symmetry  (ID’s 
57,13,17,50,  and  86)  were  not.  If  this  50%  threshold  is  adopted,  which  is  supported  based  on 
results  from  the  emplaced  targets  (Table  5),  the  cued  results  would  accurately  predict 
symmetry  for  12  of  the  13  objects.  As  noted  above,  ID#  14  consists  of  two  objects  buried  in 
close  proximity,  thereby  violating  the  single-source  assumption  of  the  inversion  routines. 
Three  of  the  cued  measurements  (Id’s  15,  55,  and  51)  possess  asymmetry  values  greater  than 
100%  because  one  or  more  of  the  inverted  response  coefficients  for  that  target  is  negative. 


Table  6.  Fit  Results  for  Native,  Non-emplaced  Targets;  Cued  vs.  Sky  Survey  Data 


Cued  Mode  Fit  Results 

Survey  Mode  Fit  Results 

ID 

Photograph 

ID 

Object 

Axial 

Asymmetry 

Fit  Error 
sqrt[1-(rA2)] 

Aspect  Ratio 
[B1/avg(B2,B3)] 

Asymmetry 

[(B2-B3)/B2] 

Fit  Error 
sqrt[1-(rA2)] 

Aspect  Ratio 
[B1/avg(B2,B3)j 

Asymmetry 

[(B2-B3)/B2] 

78 

87 

metal  hinge 

Yes 

0.054 

10.8 

82% 

0.438 

2.8 

58% 

60 

92 

surface  metal 

Yes 

0.070 

1.8 

68% 

0.672 

4.2 

75% 

14 

253 

2  metal  pieces 

Yes 

0.057 

10.7 

22% 

0.261 

5.6 

53% 

281 

nails 

Yes 

0.221 

3.9 

81% 

0.372 

4.0 

26% 

15 

276 

nail  and 
klinker 

Yes 

0.247 

7.7 

116% 

0.327 

4.0 

55% 

55 

109 

concrete  slab 

Yes 

0.014 

-1178.0 

205% 

0.178 

6.3 

100% 

51 

126 

2"x14"  flat 
steel  bar 

Yes 

0.057 

-2.1 

-568% 

0.385 

4.4 

100% 

57 

104 

1"  steel  pipe  in 
concrete 

No 

0.014 

4.3 

40% 

0.281 

3.2 

100% 

13 

261 

surface  metal 

No 

0.050 

13.6 

33% 

0.302 

7.0 

100% 

17 

327 

rebar 

No 

0.157 

3.5 

40% 

0.320 

3.2 

29% 

50 

387 

oil  filter 

No 

0.000 

1.5 

35% 

0.126 

2.4 

28% 

200 

356 

20in  steel  rod 

No 

0.178 

4.1 

36% 

0.267 

4.4 

79% 

86 

463 

6in  metal 
chain 

No 

0.118 

3.2 

50% 

0.150 

6.4 

74% 

MEAN  0.095  0.314 

STD  DEV  0.081  0.141 
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Figure  25.  Photographs  of  native,  non-emplaced  objects  for  which  data  was  acquired  using  a 
cued  deployment  and  production  survey.  The  photograph  ID  numbers  correspond  to  those 
listed  in  Table  6.  Although  difficult  to  read,  the  top  left  photo  is  ID  87. 
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4. 3. 2.2  Cued  data  Discrimination  Performance 


Receiver  operating  characteristic  curves  for  all  (emplaced  and  native)  cued  data  are  shown  in 
Figure  26.  In  this  case,  a  GLRT  classifier  was  trained  and  tested  on  the  same  set  of  features 
derived  exclusively  from  cued  data.  As  such,  it  presents  the  best  case  scenario  given  these 
cued  measurements. 

The  graph  on  the  left  shows  performances  if  we  define  the  target  of  interest  as  UXO.  The 
graph  on  the  right  presents  performance  if  we  define  our  target  of  interest  as  being 
symmetrical  regardless  of  its  classification  of  UXO.  In  these  graphs,  the  ‘B1  B2  B3’ 
descriptor  indicates  the  three  response  coefficients  (eigenvalues  of  the  magnetic 
polarizability  tensor)  directly.  The  term  ‘asymmetry’  and  ‘aspect  ratio’  were  defined  as 
before  (i.e.,  see  Table  6). 

UXO  Symmetry  Metric 


Figure  26.  Receiver  operating  characteristic  curves  for  cued  data  only. 

4. 3. 2. 3  Controlled  Test  Measurements 

For  comparison  purposes,  Table  7  presents  estimated  shape  information  as  derived  from  the 
test  stand  data.  In  this  table,  the  mean  value  and  standard  deviation  are  reported  for  the 
coherence,  estimated  aspect  ratio,  and  estimated  asymmetry.  Statistical  measures  are 
reported  because  the  test  stand  protocol  included  recording  data  for  each  item  at  six  discrete 
orientations  relative  to  the  sensor  coil.  The  sensor-ordnance  orientations  were  nose  up,  nose 
down,  horizontal  along  the  X-axis,  horizontal  along  Y-axis,  45-degree  nose  up,  and  45-degre 
nose  down.  The  ordnance  objects  were  positioned  such  that  the  signal  strength  was  as  strong 
as  possible  without  saturating  the  analog-to-digital  converter.  A  couple  of  interesting  notes 
are  apparent.  First,  by  looking  at  the  fit  errors,  it  is  clear  that  the  inversion  process 
reproduced  the  measured  data  very  well.  Second,  and  perhaps  most  importantly,  the  mean 
asymmetry  values  for  emplaced,  symmetrical  UXO  is  15%  compared  to  25%  for  the  cued 
and  60%  for  Sky’s  survey  data  (Table  5).  In  other  words,  the  inverted  shape  parameters 
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more  accurately  and  consistently  reflect  axial  symmetry  than  either  the  cued  or  production 
survey  data. 


Table  7.  Fit  Results  for  Emplaced  Targets;  Test  Stand  Data 


Test  Stand  Fit  Results 

Object 

Fit  Error 
sqrt[1-(rA2)] 

Aspect  Ratio 
[B1/avg(B2,B3)j 

Asymmetry 

[(B2-B3)/B2] 

Mean 

Std  Deviation 

Mean 

Std  Deviation 

Mean 

Std  Deviation 

37mm  Projectile 

0.068 

0.017 

3.8 

0.69 

10% 

5% 

40mm  Grenade 

0.199 

0.023 

1.7 

0.30 

33% 

14% 

40mm  Projectile 

0.090 

0.051 

2.5 

0.56 

13% 

7% 

57mm  Projectile 

0.052 

0.028 

1.9 

0.32 

9% 

5% 

60mm  Projectile 

0.036 

0.017 

3.8 

0.57 

11% 

6% 

105mm  Projectile 

0.019 

0.009 

1.9 

0.62 

6% 

3% 

2.75in  Rocket 

0.020 

0.013 

7.4 

1.38 

22% 

17% 

MEAN  0.069  15% 


4. 3. 2.4  Sky  EM61  Survey  Data 

A  total  of  150  anomalies  were  identified  in  the  production  EMI  data  (shown  in  Figure  19) 
and  inverted  for  model  parameters.  The  model  parameters  were  classified  using  a  support 
vector  machine  (SVM).  The  features  used  by  the  SVM  classifier  were  (i)  the  estimated  size, 
(ii)  the  two  smaller  response  coefficients,  and  (iii)  coherence.  For  this  effort,  we  ranked  the 
classifier  outputs  using  a  scheme  of  1,  2,  and  3,  where  a  rank  of  1  indicates  high  confidence 
UXO,  rank  3  is  used  for  high  confidence  clutter,  and  rank  2  is  everything  else.  The  rankings 
were  selected  after  plotting  the  classifier  outputs  in  descending  order  and  superimposing 
classifier  outputs  for  the  seven  calibration  targets  (Figure  27).  A  threshold  of  0.25  was 
selected  to  separate  rankings  1  and  2,  and  a  threshold  of  -0.2  was  used  to  segment  rankings  of 
2  and  3.  The  later  threshold  was  partially  based  on  the  slope  change  in  the  classifier  output. 

Results  of  the  classification  reveal  that  of  the  58  seeded  items,  41  were  ranked  as  high 
confidence  ordnance  rank  1,10  were  ranked  as  high  confidence  clutter  rank  3,  and  7 
remained  unclassified  (rank  2).  Figure  28  presents  a  receiver  operating  characteristic  curve. 
From  the  ROC,  we  see  that  the  %  UXO  correctly  classified  is  0.74  (viz.,  the  Pd  for  rank  1) 
and  the  False  Alarm  metric  is  0. 15  (1-Pd  for  rank  3  anomalies).  Note  that  the  ROC  includes 
results  for  the  seven  calibration  targets  that  were  within  the  survey  area. 
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Dupont  EMI:  SVM  Classifier 
Features  used:  fitted  size,  beta-2,  beta-3,  coherence 


Figure  27.  Output  from  Duke’s  Support  Vector  Machine  classifier;  Sky’s  EM61  data.  Each 
point  represents  the  classifier  value  for  an  individual  anomaly.  The  magenta  symbols 
identify  inert  UXO  items  buried  within  the  site  which  were  used  for  calibration  purposes. 
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EM61  Survey  Data  ROC  -  Lake  Success  Business  Park 


Number  of  False  Alarms 


Figure  28.  Receiver  operating  characteristic  curve  for  EMI  classifications.  The  blue  symbols 
plot  Pd  as  a  function  of  the  number  of  false  alarms.  The  red  diamonds  identify  thresholds 
between  rankings. 

4. 3. 2. 5  Observed  Fit  Errors;  dynamic  versus  static  surveying 

For  comparison  purposes,  Figure  29  overlays  the  model  fit  errors  for  data  acquired  in  a 
moving,  dynamic  sensor  platform  (Sky’s  EM61  survey  data)  with  those  associated  with  a 
non-moving,  stationary  sampling  scheme  (cued  and  test  stand  data).  As  clearly  observed 
here,  the  fit  errors  for  the  dynamic  survey  are  significantly  larger  than  either  of  the  non¬ 
moving  acquisition  modes.  Note  that  although  most  of  the  cued  results  are  grouped  with  the 
test  stand  results,  some  are  mingled  with  the  survey  fits.  After  investigating,  we  find  that 
larger  fit  errors  for  the  cued  data  were  caused  by  overlapping  signatures  or  were  associated 
with  low  amplitude  responses. 
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Comparison  of  Fit  Error  for  Test  Stand, 
Cued,  and  Survey  deployments 


Figure  29.  Scatter  plot  of  the  maximum  signal  versus  dipole  fit  error  for  the  test  stand 
measurements,  all  cued  targets  (emplaced  and  non-emplaced),  and  all  anomalies  detected  in 
Sky’s  EM61  MKII  survey. 

4.3.3  Depth  Estimates 

Comparisons  of  the  actual  versus  fitted  depth  are  shown  in  Figure  30  for  cued  and  survey 
mode  EMI  data.  The  top  plot  shows  results  for  all  of  the  emplaced  targets  regardless  of  the 
fit  error  between  measured  and  modeled  data.  As  anticipated,  the  cued  data  depth  estimates 
show  less  scatter  and  better  match  actual  depths  than  the  survey  data.  The  bottom  plot  only 
includes  results  for  objects  that  posses  a  fit  error  of  less  than  0.22  (selected  here  for 
illustrative  purposes).  Comparing  the  two  plots,  we  see  that  all  of  the  cued  features  remain 
but  a  significant  number  of  the  survey  data  features  were  removed.  In  fact,  the  mean  fit  error 
for  cued  data  collected  over  the  emplaced  UXO  was  0.09  compared  to  0.37  for  the  survey 
data.  Also  note  that  most  of  the  cases  above  this  threshold  also  had  inaccurate  depth 
estimates. 
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Figure  30.  Actual  versus  fitted  depths  for  emplaced  targets.  Top:  all  seeded  items,  bottom: 
results  for  anomalies  that  possess  fit  errors  of  less  than  0.22. 
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4.3.4  Qualitative  Metrics 

UX- Analyze  characterizes  and  classifies  individual  anomalies.  Steps  taken  within  the 
characterization  phase  include  identifying  anomalies,  selecting  the  spatial  footprint  of  the 
anomaly,  and  reviewing  the  results.  The  most  labor  intensive  portion  of  the  characterization 
phase  is  identifying  the  spatial  extent  of  each  anomaly  and  editing  individual  samples  that  are 
inconsistent.  UX- Analyze  provides  graphical  presentations  and  appropriate  tools  to  assist  the 
analyst.  Once  completed,  individual  anomalies  are  inverted  manually  or  in  batch  mode.  The 
output  of  the  characterization  phase  is  a  list  of  model  parameters  for  each  anomaly  which  is 
then  submitted  to  the  classifier.  Steps  taken  during  the  classification  phase  include  creating 
or  updating  a  library  for  training  purposes,  selecting  and  calling  the  classifier. 

UX- Analyze  adheres  to  the  look  and  feel  of  Oasis  montaj  and,  as  such,  is  quickly  mastered 
by  experienced  Oasis  montaj  users.  It  is  easy  to  use  and  includes  an  on-line  help  menu. 
During  this  demonstration,  Oasis  montaj  underwent  a  number  of  major  upgrades  that  affected 
the  performance  and  robustness  of  UX- Analyze.  These  coding  inconsistencies  have  been 
corrected.  As  of  this  report,  January  2007,  the  current  Oasis  montaj  release  is  6.4,  and  we  do 
not  experience  systematic  difficulties  while  processing  and  analyzing  these  magnetic  and 
EMI  data. 


4.4  Discussion 

The  feature-based  EMI  characterization  approach  demonstrated  here  inverts  measured  EM61 
field  data  assuming  a  point  dipole  source  and  then  separates  the  targets  into  classes  using  the 
model  parameters.  The  derived  model  parameters  are  meaningful,  however,  only  if  the 
modeled  data  accurately  represent  the  measured  data.  In  other  words,  the  dipole  fit  error,  or 
the  difference  between  the  measured  and  modeled  data,  must  be  insignificant.  We  have 
found  through  experience  that  if  the  fit  errors  are  greater  than  0.1,  the  recovered  parameters 
are  often  inaccurate. 

Unfortunately,  the  mean  fit  error  for  EM61  survey  data  acquired  during  this  demonstration  is 
0.3  or  larger  (survey  mode  emplaced  targets  only;  Table  5).  In  other  words,  for  the  average 
anomaly,  only  70  percent  of  the  measured  data  is  explained  by  the  model.  Because  of  this, 
little  confidence  can  be  placed  in  the  inverted  features.  These  EM61  survey  data,  which  were 
spatially  registered  using  a  laser  based  robotic  total  station,  do  not  support  feature  based 
decisions. 

In  contrast  to  the  survey  data,  the  cued  measurements  produced  data  that  was  well  modeled 
using  a  dipole  source.  On  average,  the  inversion  process  accurately  accounted  for  over  94% 
of  each  cued  anomaly  data.  Even  given  the  good  model  fits,  however,  perfect  classification 
isn’t  guaranteed.  For  these  cued  data,  the  discrimination  performance  is  along  the  chance 
diagonal  if  the  inverted,  intrinsic  shape  parameters  (viz.,  the  three  response  coefficients)  are 
used  to  classify  targets  as  UXO  or  not  (Figure  26).  Performances  are  improved,  but  not  yet 
ideal,  if  we  classify  using  ratios  of  the  betas,  axial  asymmetry  and/or  aspect  ratio  estimates. 
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Additional  performance  gains  are  realized  if  we  decide  that  our  target  of  interest  includes 
axially  symmetric  metallic  objects  as  well  as  UXO. 

5.  Cost  Assessment 


5.1  Cost  Reporting 


Cost  Category 

Sub  Category 

Costs 

Data  Acquisition 

Site  Visit,  Equipment  Prep 
and  Survey 

$40,000 

Data  Characterization  and 
Classification 

Anomaly  Specific 
Characterization 

250  @  $6  per  anomaly 

$1,500 

Anomaly  Plots 

250  @  $0.05  per  anomaly 

$15 

Reporting 

Analysis  &  Final  Report 

$60,000 

5.2  Cost  Analysis 

The  major  cost  components  of  this  demonstration  acquiring  the  data,  making  sense  of  the 
results,  and  compiling  the  report.  A  non-trivial  amount  of  labor  and  effort  was  exerted  trying 
to  acquire  high-fidelity  data  that  would  support  feature-based  analysis,  process  the  data  once 
acquired,  and  to  extract  all  possible  information  from  the  signatures.  As  shown  above,  the 
data  characterization  and  classification  tasks  were  not  major  contributions  to  the  project 
costs. 


6.  Implementation  Issues 


6.1  End  User  Issues 

The  primary  end-users  of  this  data  analysis  technology  will  likely  be  high-end  geophysical 
service  providers,  technically  savvy  DoD  oversight  personnel,  and  government-sponsored 
researchers.  As  demonstrated  here,  however,  feature-based  discrimination  decisions  require 
geophysical  data  that  have  high  signal-to-noise  ratios,  are  isolated,  and  possess  high  spatial 
registration  precision. 
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